Sequentially overlaying media content

ABSTRACT

A method, system, or device to determine when media content has been displayed. The method can include receiving a unique identifier (UID) and fingerprint for a media segment. The method can also include determining a media segment ID for the media segment, generating and submitting a query for the media segment ID in a record database, determining that the media segment ID is not stored in the record database, and identifying an overlay content segment ID. The method can include generating and submitting for the overlay content segment ID in the record database, determining it is not stored in an entry, sending the overlay content segment to the electronic device, and receiving a notification from the electronic device.

CROSS REFERENCE TO RELATED APPLICATIONS

This U.S. patent application is a continuation of, and claims priorityunder 35 U.S.C. § 120 from U.S. patent application Ser. No. 15/842,750,filed on Dec. 14, 2017, which is a continuation of, and claims priorityunder 35 U.S.C. § 120 from U.S. patent application Ser. No. 15/130,823,filed on Apr. 15, 2016, which claims priority under 35 U.S.C. § 119(e)to U.S. Provisional Application 62/235,811, filed Oct. 1, 2015 and U.S.Provisional Application No. 62/235,856, filed Oct. 1, 2015. Thedisclosures of these prior applications are considered part of thedisclosure of this application and are hereby incorporated by referencein their entireties.

BACKGROUND

Media consumption devices, such as smart televisions (TVs), may accessbroadcast digital content and receive data, such as streaming media,from data networks. Streaming media refers to a service in which an enduser may request media content, such as movies or news, over a telephoneline, a cable channel, an Internet channel, and so forth. For example, auser may view a movie without having to leave their residence. Inaddition, users may access various types of educational content, such asvideo lectures, without having to physically attend a class at a schoolor an educational institution.

As the number of media consumption devices continues to increase, videocontent generation and delivery may similarly increase. With an increasein use of media consumption devices that access the broadcast digitalcontent and the streaming media, content providers or network providersmay distribute contextually relevant material to viewers that areconsuming the broadcast digital content or the streaming media. Forexample, local broadcasters may include contextually relevantadvertisements and interactive content with the broadcast digitalcontent or the streaming media.

DESCRIPTION OF DRAWINGS

The disclosure will be understood more fully from the detaileddescription given below and from the accompanying drawings of variousembodiments of the disclosure. The drawings, however, are not to betaken to limit the disclosure to the specific embodiments, but are forexplanation and understanding only.

FIG. 1 illustrates a system diagram of a content distribution networkaccording to one embodiment.

FIG. 2A illustrates a system diagram of a content overlay networkaccording to one embodiment.

FIG. 2B illustrates a system diagram of a content overlay networkaccording to one embodiment.

FIG. 3A illustrates a diagram of a method for determining when a mediasegment has previously been displayed on an electronic device accordingto one embodiment

FIG. 3B illustrates a diagram of a method for determining when toreplace a media segment that has previously been displayed on anelectronic device according to one embodiment

FIG. 4 illustrates a diagrammatic representation of a machine in theexample form of a computer system within which a set of instructions,for causing the machine to perform any one or more of the methodologiesdiscussed herein, may be executed.

FIG. 5 illustrates a diagram of a method for determining when to replacea multimedia item that has previously been displayed on an electronicdevice according to one embodiment.

FIG. 6 illustrates a diagram of a method of checking when a multimediaitem has previously been displayed on an electronic device according toone embodiment.

DETAILED DESCRIPTION

Currently, there are several hundred program choices offered by cabletelevision (TV) providers and satellite providers. In addition, thereare over one hundred major television market areas with dozens of localtelevision channels. A content provider or broadcaster may generatecontent feeds at a central location and distribute the content feeds toa variety of locations. For example, a content provider may distribute atelevision channel to a number of network providers. The networkproviders can be local broadcasters, multi-channel networks, and othercontent owners or distributors.

The word “content” can refer to media or multimedia. The word “content”can also be a specific term that means the subject matter of the mediumrather than the medium itself. Likewise, the word “media” and somecompound words that include “media” (e.g. multimedia, hypermedia) areinstead referring to content, rather than to the channel through whichthe information is delivered to the end user/audience. An example of atype of content commonly referred to as a type of media is a “motionpicture,” referred to as “a film.” Another example is a “video,” whichtypically includes video frames and audio samples.

Media content broadcasting or streaming, such as TV show or internetbroadcast, may be an engaging venue to advertise products and services,provide information to viewers, or any combination thereof. To createrevenue to provide numerous programming choices, content providers (suchas television networks) can insert advertisements (ADs) into mediacontent. Conventionally, advertisers have relied on sending one videostream to all the viewers of a channel, with the only targeting beingperformed by the local broadcasters based on an audience analysis andratings providers, such as ratings provided by The Nielsen Company®.

Broadcasters, content providers, and advertisers want to know whatprograms or channels individual viewers are watching. The broadcasters,content providers, and advertisers may also want to know when theindividual viewers are watching the programs or channels. Thebroadcasters, content providers, and advertisers can use thisinformation to target subject matter and a timing of ADs or commercialsto viewers. For example, broadcasters want to target ADs and othernon-commercial information to specific viewers. Non-commercial mayinclude news alerts, announcements, educational information, and soforth.

While TV advertising is an engaging venue to advertise products orservices, conventional TV advertising lacks targeted and personalizedadvertising. Additionally, conventional TV advertising does not enableretargeting or further engagement of viewers once the viewers havewatched a commercial. For example, to reach a viewer with a TV AD,conventional TV advertising assumes a viewer is watching the samechannel for a period of time. In this example, a content provider oradvertiser will show the TV AD within the period of time. In anotherexample, to reach a viewer with the TV AD, a content provider oradvertiser will use conventional TV advertising and show the TV ADrepeatedly over a period of time. In this example, the content provideror the advertiser assumes the viewer will see the TV AD within theperiod of time. However, these assumptions are often incorrect and cancause a viewer to miss viewing the TV AD at all. Alternatively, theseassumptions can cause viewer to see the same TV AD multiple times, butthe multiple viewings can have a negligible effect on the customer'sintent to purchase a product shown in the TV AD. It would therefore beadvantageous to determine what program a viewer is watching or is aboutto watch and send an identifier correlating to the program informationto an overlay content server or an electronic device. The electronicdevice or overlay content server can use the identifier to target ADs toa viewer.

To make this determination, an ACR system can generate queryfingerprints from media content (e.g., a media program or channel) auser is watching using automatic content recognition (ACR). The ACRsystem may compare query fingerprints with stored fingerprints within adatabase to determine a match. This database can be stored at a server.In one example, the server can be an ACR server.

The ACR system can send the query fingerprints to the server. The servercan find one or more matches between the query fingerprints and thestored fingerprints. In one example, when the server finds a match, theserver can identify a channel that a TV program is currently beingstreamed on. In another example, when the server finds a match, theserver can identify the channel that the TV program is going to bestreamed on.

The ACR server may then send a message to a requesting entity that isrequesting an identification of the media content. The requesting entitymay be an ACR system integrated into a smart TV or the overlay contentserver. The smart television can include a processing device and radiofrequency (RF) circuitry. The processing device can execute an operatingsystem (OS) and/or one or more applications, such as applications toaccess video-on-demand (VOD) content. The processing device can use anantenna of the radio frequency (RF) circuitry to communicate on awireless local area network (WLAN). In one example, the processingdevice can use the RF circuitry to connect to the internet to accessonline interactive media or on-demand streaming media.

The requesting entity may use the identified media content to selectoverlay content that can be displayed to a user. The requesting entitymay replace or overlay the different overlay content on top of the mediacontent an electronic device is currently displaying to the viewer. Inone example, the requesting entity can replace or overlay the mediacontent the electronic device is currently displaying to the viewerduring a commercial break with overlay content. In another example, therequesting entity may replace or overlay the media content theelectronic device is displaying to the viewer during a regularlyscheduled program with overlay content.

Because knowledge of the channel is time sensitive as commercial timeslots change with different channels, an requesting entity can (such asa TV or set-top box) queries the server continuously or periodically. Inone example, the ACR system that is integrated into the electronicdevice can query the server every second. In another example, the ACRsystem can query the server every several seconds. Conventionally, anACR system fingerprints media content from the content feed and areference database stores the fingerprints. The ACR system of theelectronic device may perform local ACR fingerprint matching using thereference database. For example, the media content fingerprinting occursat a broadcast facility and then the ACR system of the electronic deviceperforms local ACR fingerprint matching. However, the broadcast facilityperforming the fingerprinting and the electronic device performing theACR fingerprint matching can result in detecting ADs and other mediacontent only at the time of viewing without any advanced detection.

Described herein are methods, systems, and devices for matchingfingerprints stored in a reference database with fingerprints generatedfrom media content a user is watching while indexing the existingfingerprints in a reference database. By matching the fingerprints whileindexing the existing fingerprints, an ACR system of the electronicdevice can detect content segments within other media content. In oneexample, the electronic device can index a live broadcast feed severalseconds before the electronic device displays the live broadcast feed.One advantage of matching during indexing is to alert an electronicdevice of upcoming content. Another advantage of matching duringindexing is to provide the electronic device analytics about how manychannels and which channels have aired specific commercials or videoclips.

In one example, an electronic device may include a computer-readablestorage medium storing instructions. The computer-readable storagemedium can be non-transitory. The electronic device can includeprocessing logic to execute the instructions to generate a firstfingerprint for media content a user is consuming. In another example, amethod may include storing instructions on the computer-readable storagemedium and executing the instructions by the processing logic togenerate a query fingerprint for media content a user is consuming. Theprocessing logic may include hardware (e.g., circuitry, dedicated logic,programmable logic, microcode, etc.), software (e.g., instructionsexecuted by a processing device), firmware or a combination thereof. Theelectronic device can send the query fingerprint to a server. The servercan send an identifier for a first channel a viewer is watching to theelectronic device in response to the server finding a match between thequery fingerprint and a media content fingerprint in a database. Theelectronic device may continuously receive multiple fingerprints fromthe server that are generated from the media content in locations in abroadcast chain that proceed where the user will view the media content.The electronic device can store the multiple fingerprints in a localdatabase of the computer-readable storage medium.

The electronic device or server may send an identification of thechannel to a look-up server that identifies the media content the useris watching. The look-up server may then send an identifier to anoverlay content server indicating what media content the electronicdevice is displaying. The overlay content server can send media contentto a look-up server, where the media content is contextually relevant toa subject matter of the media content. The overlay content server maydeliver the media content to the electronic device for display asoverlay content. In one embodiment, the overlay content can be mediacontent that an ACR system overlays or superimposes on top of mediacontent an electronic device is displaying to a viewer. In anotherembodiment, the overlay content can be media content that replaces mediacontent an electronic device is displaying to a viewer, e.g.,replacement media content. In one example, the overlay content servercan store the overlay content. In another example, the overlay contentserver can be coupled to a memory device (such as another server) thatcan store the overlay content. In this example, the overlay contentserver can retrieve the overlay content from the memory device and sendit to the electronic device. Alternatively, the ACR system can overlaythe overlay content on top of a portion of the media content theelectronic device is currently displaying to the viewer. For example,overlay content may be overlayed on top of a media content that refersto the subject matter of the media content.

To provide viewers with media content, an individual or an organizationmay decide to stream the media content to viewers, such as by deliveringthe media content over the Internet to the viewers. The media contentused by the individual or the organization may be media content acquiredfrom one or more live broadcast media feeds. For example, a mediacontent provider may provide a user with a linear media channel. Thelinear media channel can be a live media feed source provided to aviewer over the Internet.

Described herein are methods, systems, and devices for limiting a numberof times a multimedia item is displayed. When a first multimedia itemhas been displayed a threshold number of times, a second multimedia itemcan replace the first multimedia item. For example, processing logic ofan electronic device can overlay the second multimedia item on to thefirst multimedia item to replace the first multimedia item. Theprocessing logic may include hardware (e.g., circuitry, dedicated logic,programmable logic, microcode, etc.), software (e.g., instructionsexecuted by a processing device), firmware or a combination thereof. Inanother example, the first multimedia item and the second multimediaitem are multimedia items in a multimedia content set that includesmultiple multimedia content items. Each time an individual has seen afirst multimedia item on an electronic device, a record can be made ofthe viewing of the multimedia item on the electronic device. When anumber of times the first multimedia item is to be displayed on theelectronic device is exceeded, the processing logic of the electronicdevice can replace the first multimedia item with a second multimediaitem in the multimedia content set.

One advantage of replacing the first multimedia item with the secondmultimedia item can be to eliminate over-exposing or under-exposing aviewer to the first multimedia item. For example, replacing the firstmultimedia item with the second multimedia item avoids showing the sameperson the same multimedia item an excessive number of times. Displayingthe same multimedia item an excessive number of times has a diminishingreturn on investment (ROI) for displaying the AD. Another advantage ofreplacing the first multimedia item with the second multimedia item canbe to sequentially display multimedia items in the multimedia contentset to the viewer. For example, the electronic device can sequentiallydisplay the multimedia items of the multimedia content set to provide aprogressive storyline to a viewer over multiple viewings by the user onthe electronic device. Another advantage of replacing the firstmultimedia item with the second multimedia item can be to reduce anumber of times or a period of time a content provider or advertisershows the first multimedia item to achieve a desired confidence levelthat the AD has been viewed by the viewer. For example, once the firstmultimedia item has been displayed to the viewer the second multimediaitem in the multimedia content set can be displayed to the view toreduce advertising costs to an advertiser. The advertising cost can bereduced by decreasing a number of advertising spaces the advertiserneeds to buy and reducing a risk that the viewer will not see the firstmultimedia item of the multimedia content set. The multimedia items ofthe multimedia content set can be overlay content that includes: ADs,news alerts, announcements, or other information in addition to mediacontent of a content feed that is displayed to the viewers.

Media content of a content feed can be matched with the overlay contentusing an overlay scheme. In one example, the overlay scheme candecompose the media content into a series of frames or sections using anautomatic content recognition (ACR) device or technology. In anotherexample, the frames or sections of the media content can include tags oridentifiers used to match the frames or sections with the overlaycontent. For example, the tags or identifiers can include informationdescribing content of the frames or sections for matching with theoverlay content.

To enhance broadcast or streamed content, the electronic devicedisplaying the media content form the content feed can identify themedia content using the ACR system or technology. When the ACR systemhas identified the media content, the ACR system can provide overlaycontent to superimpose onto the media content. The media content can betextual, graphical, image or video content from a content distributionnetwork.

FIG. 1 illustrates a system diagram of a content distribution network100 according to one embodiment. In the content distribution network100, a content provider 102 may broadcast a content feed to a localprovider 106. The local provider 106 may include a headend 104 and anACR fingerprinter server 105. The headend 104 of the local provider 106can receive a content feed from the content provider 102. The headend104 may generate a local content feed based on the received contentfeed. For example, the headend 104 may be a local affiliate broadcasterreceiving a network channel with programming and advertisements from anational or global broadcaster. In one example, the headend 104 can be asource of a broadcast feed prior to the broadcast facility transmittingthe broadcast feed. In another example, the headend 104 can be adistribution amplifier. The distribution amplifier can receive a sourcefeed and it can create multiple outputs for different devices from thesame source feed. The output feeds can be routed to various distributionoutlets, such as for broadcast over the air (OTA), delivery to cableproviders, delivery to satellite providers, and/or delivery to onlinestreaming providers.

The headend 104 may communicate the local content feed to the ACRfingerprinter server 105, an over-the-air (OTA) broadcaster 108, and/ora multichannel video programming distributor (MVPD) 110. In one example,the ACR fingerprinter server 105 can be a device that is installed in abroadcast facility. The ACR fingerprinter server 105 can receive a feedfrom the headend 104. The ACR fingerprinter server 105 can fingerprintthe feed and send the fingerprints to a cloud-based web service. In oneembodiment, the ACR fingerprinter server 105 may be installed inside thebroadcast facility in order to fingerprint a feed in advance of thatfeed being delivered to a display device, such as a television (TV).

The OTA broadcaster 108 and/or the MVPD 110 may communicate the localcontent feed to electronic devices 115. Some examples of the electronicdevices 115 include electronic devices 118 and 120, a set top box 114that streams media content to the electronic devices 118 and 120, aswell as other devices 116 through which the user may stream the localcontent feed, e.g., wirelessly.

In one example, the OTA broadcaster 108 may broadcast the local contentfeed using traditional local television channels or radio channels. Inthis example, the electronic devices 118 and 120 may include antennas(such as TV antennas or radio antennas) to receive the local contentfeed. In another example, the MVPD 110 (such as a cable broadcaster or asatellite broadcaster) may communicate the local content feed to a settop box 114. In this example, the set top box 114 may format the contentfeed for the electronic devices 118 and 120 and may communicate theformatted content feed to the electronic devices 118 and 120. Theelectronic devices 118 and 120 may include a display device, such as atelevision screen or a touch screen, to display the local content to aviewer. Various components of the content distribution network 100 maybe integrated or coupled to the electronic devices 118 and 120. Forexample, the electronic devices 118 and 120 can be smart televisionsthat include the antennas, the set top box 114, and the display devicein a single unit.

The ACR fingerprint server 105 may analyze the local content feed anddetermine fingerprint information (e.g., fingerprints). The ACRfingerprint server 105 may communicate the fingerprints to the ACRsystem 124 or 126. The ACR systems 124 and 126 may be different ACRsystems selected by device manufacturers, such as smart TVmanufacturers. In one example, the ACR system 124 or 126 can includeprocessing logic or a processing device to execute instructions orperform processes. In another example, the ACR systems 124 and 126 canbe ACR web services for different ACR vendors. The ACR fingerprinterserver 105 can enable the use of indexing clients from multiple ACRvendors. For example, different consumer electronics manufacturers(CEMs), such as TV manufactures, can independently select ACR vendorsfor integration into their electronic devices. To enable ACR-basedfunctionality on a specific CEM's device, an ACR system can indexbroadcast feeds using ACR vendors CEMs have selected and integrated onthat CEM's televisions. One advantage of integrating multiple ACRvendors onto the ACR fingerprinter server 105 can be to enable entitiessuch as advertisers, advertising agencies, and content providers tointerface with a single platform and deliver advanced advertising andcontent experiences to electronic devices from multiple CEMs.

In one example, the ACR system 124 can be coupled to a referencedatabase 125. In another example, the ACR system 126 can be coupled to areference database 127. The reference databases 125 and 127 can storefingerprints and media content. The content manager 122 can be a servercomponent that receives messages from the ACR system 124 and/or 126. Themessages can include information indicating media content that theelectronic device 118 or 120 is going to display or is currentlydisplaying. In one example, in response to the content manager 122receiving the message, the content manager 122 can record an impressionor association between the electronic device 118 or 120 and the mediacontent stored in the database 123. In another example, in response tothe content manager 122 receiving the message, the content manager 122can generate and submit a query or search of the database 123 foravailable overlay content for the media content. The content manager 122filters the overlay content using the device information for theelectronic device 118 or 120. The content manager 122 can send anoverlay content, via the ACR system 124 or 126, to the electronic device118 or 120. In one embodiment, the electronic device 118 or 120 candisplay the overlay content to a viewer. In another embodiment, theelectronic device 118 or 120 can replace the media content with theoverlay content and display the overlay content to a viewer. In anotherexample, the ACR system 124 or 126 can cause triggers to be displayed onthe electronic device 118 or 120. The electronic device 118 or 120 canrecognize the triggers and send a request to the content manager 122 torecord an impression or association between the electronic device 118 or120 and the media content in the database 123. For example, theelectronic device 118 or 120 can send a request to the content manager122 to record whether a viewer changes a channel when the electronicdevice 118 or 120 displays select media content.

The automatic content recognition may be performed in a distributedarchitecture as described herein. In one embodiment, the ACR client 225can reside on any of the electronic device 115, 118, or 120. In oneembodiment, the ACR client 225 can perform automatic contentrecognition. In another embodiment, a local ACR system or a local ACRfingerprint matcher integrated into can perform the automatic contentrecognition. In another embodiment, the ACR client 225 may perform ACRfingerprint matching as described herein. In another embodiment, the ACRclient 225 can be a client-side matching component.

The electronic devices 115, 118, or 120 can be a media device, such as atelevision, mobile phone, or dongle. The ACR client 225 can be softwareor firmware that executes or runs on the electronic devices 115, 118, or120. The ACR client 225 can fingerprint media content (such as audiocontent, video content, closed captions, emergency messages, and soforth) during a playback of the media content. The ACR client 225 cancommunicate with ACR system 124 or 126 to facilitate identification ofcontent and trigger events to cause the content overlay system toperform dynamic ad replacement and/or superimposing overlay content ontop of the media content. The proceeding paragraphs describe theinteractions of the ACR client 225 with the ACR fingerprint server 105in greater detail.

The ACR fingerprint server 105 may analyze the local content feed thatmay include an ordered sequence of frames from the local content feedand capture fingerprints. The ACR fingerprint server 105 may communicatethe fingerprints to the ACR system 124 or 126. The ACR systems 124 and126 may be different ACR systems selected by device manufacturers, suchas smart TV manufacturers. In one example, the ACR fingerprint server105 may format fingerprints for the different ACR systems 124 and 126.The ACR systems 124 and 126 may establish communication connections withthe different electronic devices 115. The electronic devices 115 maycommunicate fingerprint information to the ACR systems 124 and 126,respectively. When the ACR system 124 or 126 receives ACR fingerprintinformation from the electronic devices 115, the ACR system 124 or 126may match the received fingerprints with those generated by the ACRfingerprint server 105. When a match occurs and the ACR system 124 or126 identifies content, the ACR system 124 or 126 may communicate ACRevents to a content manager 122.

In another example, the ACR system 124 or 126 may receive ACRfingerprint information from one of the electronic devices 115 and maymatch the received fingerprints with those generated by the ACRfingerprint server 105. When a match occurs and the ACR system 124 or126 identifies content, the ACR system 124 or 126 may notify theelectronic device 115 of the ACR events. The electronic device 115 maycommunicate ACR event information to the content manager 122.Alternatively, or additionally, the ACR system 124 or 126 may directlycommunicate the ACR events to the content manager 122. The ACR eventfingerprint information may include: information indicating theadvertisements in the local content feed; information indicatingselected or flagged content in the local content feed; or informationindicating a change of a content channel at the electronic device 115 tonew media content.

The ACR event information from the different ACR systems 124 and 126 maybe in different formats and the content manager 122 may normalize thedata into a common format. The content manager 122 can store thenormalized data into a database 123. For example, the content manager122 may receive disparate data sets from the ACR systems 124 and 126,the disparate data sets including similar but not identical data, suchas data with the same content but formatted differently. The contentmanager 122 can process and reformat the disparate data sets to create asingle data model or format (e.g., reformatted data sets). The contentmanager 122 can store the reformatted data sets in the database 123.

In one embodiment, to normalize disparate data sets from ACR systems 124and 126, the content manager 122 may remove or filter data in the datasets. For example, some data sets may include fields or data that may beirrelevant to the content manager 122. In this example, the contentmanager 122 may remove or filter the irrelevant data. In anotherembodiment, to normalize the disparate data sets from ACR systems 124and 126, the content manager 122 may map fields of the data sets. Forexample, when the content manager 122 receives a first data set from theACR system 124 and a second data set from the ACR system 126, at leastsome of the data fields of the first data set and the second data setmay be common to both the first data set and second data set. However,the common data fields may be located at different places in the firstdata set and second data set. In this example, the content manager 122may map the different data fields of the first data set and second dataset to normalized fields and have the same data fields in the same datafield locations in the database 123.

In another embodiment, to normalize disparate data sets from the ACRsystems 124 and 126, the content manager 122 may derive data from thedata sets. For example, data from the ACR systems 124 and 126 may notinclude all of the fields that are needed to fill the data fields in thedatabase. However, the content manager 122 may use other fields in thedata sets from the ACR systems 124 and 126 to derive data for these datafields.

In one example, the database 123 may include data fields for such as astate in a country field, a designated market area (DMA), and a countyand/or city field but the data sets from the ACR systems 124 and 126 mayonly include zone improvement plan (ZIP) codes. In this example, thecontent manager 122 may use the ZIP codes to derive data for the fieldsin the database. In another example, the data set may not include anygeographic location information, but may include an internet protocol(IP) address of the ACR systems 124 and 126. In this example, thecontent manager 122 may use a geo-IP lookup service to derive the state,DMA, county, city and ZIP code information.

In another example, the database 123 may include demographic fields suchas an age field, a gender field, a household income field, and so forth.However, the data sets from the ACR systems 124 and 126 may not includethe demographic fields or demographic data. In this example, the ACRsystems 124 and 126 may provide the content manager 122 with the IPaddress of the electronic devices 115. The content manager 122 may usethe IP addresses to determine the demographic data to populate the datafields in the database.

In another example, a field in a first data set from the ACR system 124may include local time zone information, such as a mountain daylighttime (MDT) zone, and a second data set from the ACR system 126 mayinclude information from another time zone, such as a coordinateduniversal time (UTC) zone. The database may store all data using the UTCand the content manager 122 may convert the local time to UTC beforestoring the data in the database 123.

In one embodiment, the content manager 122 may use the normalized datato generate reports or data (viewing data) about user's viewing behavioracross different ACR technology vendors and smart TV or otherInternet-connected video devices. The content manager 122 and theelectronic devices 115 may include communications interfaces tocommunicate information, such as overlay content, between the electronicdevices 115 and the content manager 122. In one example, thecommunication interface may communicate the information using a cellularnetwork and/or a wireless network. In one example, the communicationnetwork may be a cellular network employing a third generationpartnership project (3GPP) release 8, 9, 10, 11, or 12 or Institute ofElectronics and Electrical Engineers, Inc. (IEEE®) 802.16p, 802.16n,802.16m-2011, 802.16h-2010, 802.16j-2009, 802.16-2009. In anotherembodiment, the communication network may be a wireless network (such asa network using the Wi-Fi® technology developed by the Wi-Fi Alliance)that may follow an IEEE® standard developed by the Institute ofElectrical and Electronics Engineers, Inc., such as the IEEE802.11-2012, IEEE 802.11ac, or IEEE 802.1 lad standards. In anotherembodiment, the communication network may deploy Bluetooth® connectionsdeveloped by the Bluetooth Special Interest Group (SIG) such asBluetooth v1.0, Bluetooth v2.0, Bluetooth v3.0, or Bluetooth v4.0. Inanother embodiment, the communication network may be a Zigbee®connection developed by the ZigBee Alliance such as IEEE 802.15.4-2003(Zigbee 2003), IEEE 802.15.4-2006 (Zigbee 2006), IEEE 802.15.4-2007(Zigbee Pro).

In one example, the content manager 122 may instruct the electronicdevices 115 to replace portions of the local content feed received fromthe OTA broadcaster 108 or the MVPD 110 with overlay content. In anotherexample, the content manager 122 may instruct the electronic devices 115to overlay or superimpose overlay content onto portions of the localcontent feed. The content manager 122 may aggregate ACR eventinformation across multiple ACR systems 124 and 126 and may communicateoverlay content to different electronic devices 115 (such as electronicdevices from different device manufacturers).

The content manager 122 may also establish communication connectionswith other devices 116. In one example, the other devices 116 maycommunicate with the electronic devices 118 and 120 and provide anadditional screen (e.g., a second screen) to display overlay content.For example, the electronic devices 118 and 120 may receive the localcontent feed from the OTA broadcaster 108 or the MVPD 110 and displaythe local content feed to a viewer. The other devices 116 may alsocommunicate ACR event information to the ACR systems 124 and 126 when anACR event occurs, as discussed in the preceding paragraphs. When thecontent manager 122 receives the ACR event information, the contentmanager 122 may communicate overlay content to the other devices 116.

In one example, the electronic devices 118 and 120 may continue todisplay the local content feed while the other devices 116 display theoverlay content. In another example, the electronic devices 118 and 120and the other devices 116 may both display the overlay content. Inanother example, the electronic devices 118 and 120 and the otherdevices 116 may display a portion of the overlay content and a portionof the local content feed. In another example, the electronic devices118 and 120 and the other devices 116 may display different localcontent feeds and different overlay content.

In one example, the electronic devices 118 and 120 and/or the otherdevices 116 may display the overlay content at the time the electronicdevices 118 and 120 and/or the other devices 116 receive the overlaycontent. In another example, the electronic devices 118 and 120 and/orthe other devices 116 may delay displaying the overlay content for athreshold period of time. In one embodiment, the threshold period oftime may be a predefined period of time. In another embodiment, thecontent manager 122 may select a period of time for the electronicdevices 118 and 120 and the other devices 116 to delay displaying theoverlay content.

FIG. 2A illustrates a system diagram of a content overlay network 200according to one embodiment. The content overlay network 200 can includean electronic device 210, an ACR system 220, a content database 230, anoverlay content server 240, a record database 250, and an overlaycontent management system 260. The ACR system 220 provides broadcastdigital content to the electronic device 210. The broadcast digitalcontent can include overlay content the ACR system 220 superimposes oroverlays on top of media segments for the electronic device 210 todisplay to a viewer. A media segment can be a segment or portion of themedia content. For example, the media segment can be a segment orportion of media content for a TV show or a movie, such as a TV show ormovie clip. In another example, the media segment of the media contentcan be a commercial that is part of media content for the TV show or themovie.

In one embodiment, the ACR system 220 is integrated into the electronicdevice 210. In another embodiment, the ACR system 220 is coupled to theelectronic device 210. In one example, the ACR system 220 can bedirectly coupled to the electronic device 210 using an input port of theelectronic device 210. For example, the ACR system 220 is aHigh-Definition Media Interface (HDMI) dongle that plugs into an HDMIinput terminal of the electronic device 210. In this example, the ACRsystem 220 can include an HDMI input port, an HDMI output port, and anetwork interface to receive broadcast content or media segments from acontent provider. In another example, the ACR system 220 can beindirectly couple to the electronic device 210 via another device thatis coupled to the electronic device 210, such as an audio and/or video(A/V) device.

The electronic device 210 can use the ACR system 220 to display targetedor personalized overlay content segment to viewers. The overlay contentsegment can be an audio, video, and/or graphical segment or clip ofcontent. In one example, the overlay content segment can be a mediaobject or file that includes an audio/video segment of content that is areplacement AD. In another example, overlay content segment is anaudio/video segment of a larger audio/video object or file that includesother overlay content segments.

In one embodiment, the ACR system 220 can use digital watermarking toACR broadcast content. For digital watermarking, a content provider or abroadcaster can process the broadcast content to insert watermark datawithin a content signal prior to broadcast the content signal. Theelectronic device 210 or the ACR system 220 can detect the watermarkdata to identify and synchronize media segments with the broadcastcontent.

In another embodiment, the ACR system 220 can identify media content ofthe broadcast content, using content fingerprinting, as the electronicdevice displays the media content. The content fingerprinting caninclude the ACR system 220 matching a sequence of content fingerprintsfor media content the electronic device 210 is displaying with ofcontent fingerprints in a fingerprint database. An advantage of contentfingerprinting can be to process the media segment while the mediasegment is being broadcasted (e.g. live processing) or within athreshold period of time, such as several seconds, after the mediasegment is broadcast (e.g. substantially live processing). The contentoverlay network 200 can adjust overlay content segments, as theelectronic device 210 displays the content, based on viewer behaviorduring a content broadcast (e.g. live or substantially liveadjustments). In one example, the viewer behavior can include theelectronic device 210 receiving a command from a user device to change achannel or to pause a display of content. In another example, the viewerbehavior can include the electronic device 210 receiving a command fromthe user device to display time-shifted content. As the viewers selector change the content, the content overlay network 200 can switch whatoverlay content is superimposed onto the media content.

The overlay content server 240 is coupled to the ACR system 220. Theoverlay content server 240 can receive overlay content from a contentprovider or advertiser. The overlay content server 240 can store mediasegments, overlay content segments, and format information for the mediasegments and overlay content segments. The overlay content server 240can send media segments to the electronic device 210 in a select overlayformat of the overlay formats.

The record database 250 is coupled to the overlay content server 240.The record database 250 can maintain a record of the media segments oroverlay content segments that the electronic device 210 displays. In oneembodiment, the electronic device 210 has a unique identifier (UID)associated with it that uniquely identifies the electronic device 210.The UID can also be associated with media segments or overlay contentsegments to indicate what media segments or overlay content segments theelectronic device 210 has displayed, as discussed in the proceedingparagraphs. In another embodiment, the record database 250 can receive amessage from the overlay content server 240. In one example, the messagecan include a media segment ID that identifies a media segment and a UIDthat is associated with the media segment ID. In another example, themessage can include an overlay content segment ID that identifies anoverlay content segment and a UID that is associated with the overlaycontent segment ID. The record database 250 can store various media andoverlay content segment IDs with their associated UIDs for subsequentlookups by the overlay content server 240. The overlay content server240 can search the record database 250 to determine when a media segmentor overlay content segment associated with a respective media segment IDor overlay content segment ID has previously been displayed on theelectronic device 210 with the UID.

In one embodiment, the overlay content management system 260 can includean input device to receive configuration information for a sequence ofmedia segments and overlay content segments. In one example, the inputdevice can be a software interface (such a web portal) on an electronicdevice, where a user can input configure information to configure thesequence of media segments and overlay content segments at the overlaycontent server 240. For example, the overlay content management system260 can be an end-user accessible website for creating, viewing,updating, and deleting overlay content and replacement videos.

In one example, the user can use the input device to set rules orcriteria for the overlay content server 240 to use to select an overlaycontent segment of the overlay content set. In one embodiment, the rulesor criteria can include a sequence of items in an overlay content set,such as media content and overlay content segments. In anotherembodiment, the rules or criteria can include displaying differentoverlay content segments in view of demographic information of the user.In one example, when the user is a male, the electronic device 210 maydisplay selected overlay content segments. In another example, when theuser is a female, other overlay content segments may be displayed. Inanother embodiment, the rules or criteria can include displayingdifferent overlay content segments in view of a time of day. Forexample, in the morning the electronic device 210 can display overlaycontent segments related to breakfast, in the evening the electronicdevice 210 can display overlay content segments related to dinner, andin the night time the electronic device 210 can display overlay contentsegments related to adult themes. In another embodiment, the overlaycontent server 240 can include an input device to receive configurationinformation to order media content and overlay content segments in alinear sequence. For example, a user can use a graphical user interface(GUI) to send configuration information to the overlay content server240 to order the sequence of media content and overlay content segments.

In another example, the overlay content management system 260 can enablea user to set up the filtering criteria which can be used by the contentmanager 122. The data that the user selects or sets up in the overlaycontent management system 260 is stored in the database 123. The overlaycontent management system 260 can enable a user to view reports relatedto the displaying of the overlay content and replacement videos.

FIG. 2B illustrates media content 278 with a media segment 280 accordingto one embodiment. Some numbers in FIG. 2B are similar to some numbersin FIG. 2A as noted by similar reference numbers, unless expresslydescribed otherwise. In one embodiment, the ACR system 220 can usefingerprinting to identify the media content 278, as discussed in thepreceding paragraphs. The media content 278 can include frames 260-276.Frames 270-274 can be the media segment 280 that is a segment of portionof the media content 278. In one embodiment, the frames 260-268 and 276can be frames in the media content 278 that are part of a TV show ormovie and the frames 270-274 and the frames 270-274 in the media segment280 can be a commercial or AD that is integrated into the TV show ormovie. In one example, the media segment 280 is a commercial that isintegrated into a TV show or movie which is broadcast or streamed to theelectronic device 210. In another example, the media segment 280 can bea segment or portion of the TV show or a movie, such as a TV show clipor movie clip. In another embodiment, the overlay content server 240 candetermine that the media segment 280 was previously displayed on theelectronic device 210 and the ACR system 220 can replace the mediasegment 280 with the overlay content segment 288, as discussed ingreater detail in the proceeding paragraphs. The overlay content segment288 can include frames 282-286. In one example, the overlay contentsegment 288 can be a replacement AD that the ACR system 220 overlays orsuperimposes on top of the frames 270-274 in the media segment 280. Inone another, the overlay content segment 288 can be a replacement ADthat the ACR system 220 replaces the frames 270-274 of media segment 280with the frames 282-286. The number of frames and the number of mediasegments in media content are not intended to be limiting. For example,the media content 278 can include multiple media segments 280 that canhave varying numbers of frames.

FIG. 3A illustrates a diagram of a method 300 for determining when anelectronic device 310 has previously displayed a media segment onaccording to one embodiment. The method 300 may at least partially beperformed by processing logic that includes hardware (e.g., circuitry,dedicated logic, programmable logic, microcode, etc.), software (e.g.,instructions executed by a processing device), firmware or a combinationthereof. The method 300 may be performed by all or part of the contentdistribution network 100 of FIG. 1 or all or part of the content overlaynetwork 200 of FIG. 2. For example, the method 300 may be performed byall or part of the processing logic of the automatic content recognition(ACR) system 124 or 126 (FIG. 1), the electronic device 115 (FIG. 1) orthe processing logic of the ACR system 220 (FIG. 2).

Referring to FIG. 3A, the method 300 begins with the electronic device310 determining a fingerprint for a media segment scheduled to bedisplayed on the electronic device 310 (312). In one example, the mediasegment can be media content the electronic device 310 receives from abroadcaster or content provider. In another example, the media segmentcan be an overlay content the electronic device 310 receives from theoverlay content server 240. In another example, the electronic device310 can have a first automatic content recognition (ACR) system and beconnected to the overlay content server via a first network connection.The electronic device 310 can receive media content from a contentsource device (such as a broadcaster) over a second network connectionbetween the content source device and the electronic device 310. In oneembodiment, a network connection can be a wired communication channel tocommunicate data, such as a communication channel via an Ethernet cableor a coaxial cable. In another embodiment, the network connection can bea wireless communication channel to communicate data, such as a cellularcommunication channel or a WLAN communication channel.

The method can also include the electronic device 310 sending a messageto the overlay content server 340 (314). The message can include thefingerprint and a UID associated with the electronic device. The methodcan include the overlay content server 340 determining a media segmentID associated with the media segment using the media segment fingerprintto identify the media segment (315). For example, the overlay contentserver 340 can search a database for a media segment. Each media segmentin the database can have a media segment ID associated with it. Themedia segment ID can include information associated with the mediasegment, such as a subject matter description of the media segment (suchas a genre or content description of the media segment), a play lengthof the media segment, a display resolution level of the media segment,and so forth. The method can include the overlay content server 340searching the record database 350 for an entry of the media segment IDassociated with the UID (316). In one embodiment, the overlay contentserver 340 can be connected to the record database 350 via a thirdnetwork connection.

The method can include the overlay content server 340 determining thatthe record database 350 is not storing the media segment ID that isassociated with the UID (318). The method can include the overlaycontent server 340 sending a notification to the electronic device 310,instructing the electronic device 310 to display the media segment(320). The method can include the electronic device 310 displaying themedia segment in response to receiving the notification (322). In oneembodiment, when the electronic device 310 does not receive anotification before the media segment is scheduled to be displayed, theelectronic device 310 will display the media segment. The method caninclude the electronic device 310 sending a notification to the overlaycontent server 340 that the media segment has been displayed on theelectronic device 310 with the associated UID (324). In one embodiment,the method can include the overlay content server 340 recording themedia segment ID and the associated UID into the record database 350 inresponse to receiving the notification that the electronic device 310has displayed the media segment (334).

In one embodiment, after the electronic device 310 has displayed themedia segment, the overlay content server 340 can determine that themedia segment has been displayed using a trigger overlay. The method caninclude the overlay content server 340 sending the trigger overlay tothe electronic device 310 when the overlay content server 340 receives anotification (326). The method can include the electronic device 310displaying the trigger overlay (328). In one embodiment, the triggeroverlay can be an invisible overlay content segment that the electronicdevice 310 displays in response to an ACR system of the electronicdevice 310 determining that the electronic device 310 has displayed aparticular media segment or is currently displaying the particular mediasegment. In another embodiment, the trigger is a web request to aspecific endpoint on the overlay content server 340 (illustrated in FIG.2). The method can include the ACR system of the electronic device 310determining that the electronic device 310 displayed the trigger overlay(330). The method can include the electronic device 310 sending anotification with the media segment ID and the UID to the overlaycontent server 340 when the electronic device 310 has displayed thetrigger (332). In one embodiment, when the ACR system of the electronicdevice is a set-top box coupled to the electronic device 310, the ACRsystem may not detect when the electronic device 310 has displayed themedia segment and can accordingly send the notification in response tothe ACR system detecting the trigger content. The method can include theoverlay content server 340 recording the media segment ID and theassociated UID into the record database 350 in response to the overlaycontent server 340 receiving the notification with the media segment IDand the UID (334).

FIG. 3B illustrates a diagram of a method 355 for determining when toreplace a media segment the electronic device 310 has previously beendisplayed according to one embodiment. The method 355 may at leastpartially be performed by processing logic that includes hardware (e.g.,circuitry, dedicated logic, programmable logic, microcode, etc.),software (e.g., instructions executed by a processing device), firmwareor a combination thereof. The method 355 may be performed by all or partof the content distribution network 100 of FIG. 1 or all or part of thecontent overlay network 200 of FIG. 2. For example, the method 355 maybe performed, at least in part, by the processing logic of the automaticcontent recognition (ACR) system 124 or 126 or the electronic device 115(FIG. 1) or the processing logic of the ACR system 220 (FIG. 2). Somenumbers in FIG. 3B are similar to some numbers in FIG. 3A as noted bysimilar reference numbers unless expressly described otherwise.

Referring to FIG. 3B, the method 355 begins with the electronic device310 determining a fingerprint for a media segment the electronic device310 is to be display (356). The method can also include the electronicdevice 310 sending a message to the overlay content server 340 (358).The message can include the fingerprint and a UID associated with theelectronic device 310. The method can include the overlay content server340 determining a media segment ID for the media segment using thefingerprint to identify the media segment (360). For example, theoverlay content server 340 can search a database for the fingerprint andidentify the media segment ID associated with the fingerprint. The mediasegment ID can include information associated with the media segment,such as a subject matter of the media segment, a play length of themedia segment, a display resolution level of the media segment, and soforth. The method can include, the overlay content server 340 searchingthe record database 350 for the media segment ID associated with the UID(362). The method can include the overlay content server 340 determiningthat the record database 350 includes an entry for the media segment IDassociated with the UID (364). When the record database 350 includes theentry of the media segment ID associated with the UID, the entry canindicate that the electronic device 310 previously displayed the firstmedia segment associated with the media segment ID.

The method can include the overlay content server 340 identifying anoverlay content segment ID for an overlay content segment in a set ofoverlay content segments (366). In one embodiment, the set of overlaycontent segments includes the overlay content segments to overlay orreplace the media segment. In another embodiment, the set of overlaycontent segments includes the media segment. The method can include theoverlay content server 340 searching the record database 350 for anentry of the overlay content segment ID (368). The method can includethe overlay content server 340 determining that the record database 350does not include the entry for the overlay content segment ID associatedwith the UID (370).

When the record database 350 does not include the entry for the overlaycontent segment ID associated with the UID, the absence of the entry canindicate that the electronic device has not previously displayed theoverlay content segment that is associated with the overlay contentsegment ID. The overlay content server 340 can store the overlay contentsegment IDs associated with the overlay content segments in the overlaycontent set. In one example, the overlay content server 340 caniteratively search the record database 350 to identify an nth overlaycontent segment that has not previously been displayed. For example,when the record database 350 includes a first entry of a first overlaycontent segment ID associated with a first overlay content segment inthe overlay content set, the overlay content server 340 can sequentiallyor randomly search the record database 350 for a second entry of asecond overlay content segment ID for a second overlay content segmentin the overlay content set, a third entry of a third overlay contentsegment ID for a third overlay content segment in the overlay contentset, a fourth entry of a fourth overlay content segment ID for a fourthoverlay content segment in the overlay content set, and so forth untilthe overlay content server 340 identifies an nth entry for an overlaycontent segment ID for an nth overlay content segment that is not storedat the overlay content server 340.

In one example, the overlay content server 340 can send the nth overlaycontent segment instead of the first overlay content segment. In anotherexample, when the record database 350 includes each of the overlaycontent segment IDs for the media and overlay content segments in theoverlay content set, the overlay content server 340 can send an overlaycontent segment from a different overlay content set to the electronicdevice 310. In this example, the content overlay network 200 can send arefund notification to a device associated with a user or administrator.For example, the user or administrator can be that an individual or arepresentative of a company, such as an advertiser that paid to haveoverlay content segments in the overlay content segment set displayed toviewers. The refund notification indicates that the overlay contentsegments in the overlay content segment set have been displayed on theelectronic device 310 prior to a threshold number of overlay slots soldto the company. In this example, the individual or the representativethat provided the initial overlay content set can receive a refund for apayment associated with the overlay content server 340 finishing sendingthe media segments and not using all of the overlay slots the individualor the representative paid for. In another example, when the recorddatabase 350 includes all of the media segment IDs for the mediasegments of the overlay content set, the overlay content server 340 canerase the media segment IDs associated with the overlay content set forthe electronic device 310 and start over in a sequence of showing themedia and overlay content segments of the overlay content set.

The method can include the overlay content server 340 sending theoverlay content segment associated with the overlay content segment ID(372). The method can include the electronic device 310 displaying theoverlay content segment (374). In one example, the electronic device 310can display the overlay content segment on the electronic device 310 byoverlaying or superimposing the overlay content segment over the mediasegment. The method can include the electronic device 310 sending anotification to overlay content server 340 that includes the overlaycontent segment ID and the UID when the electronic device 310 hasdisplayed the overlay content segment (376).

The method can include the overlay content server 340 recording an entryfor the overlay content segment ID and the associated UID into therecord database 350 in response to the overlay content server 340receiving the notification (378). Alternatively, the overlay contentserver 340 can record an entry for the overlay content segment ID andthe associated UID into the record database 350 when the overlay contentserver 340 sends the overlay content segment to the electronic device310. The preceding example illustrates the electronic device 310displaying an overlay content segment. However, this exemplary method isnot intended to be limiting. In one example, the method 355 can be aniterative method where multiple overlay content segments or all of theoverlay content segments in an overlay content set can be sequentiallyoverlayed over previously displayed media segments.

Additionally, an order of the media and overlay content segments in theoverlay content set can be sequential, random, or any other order todisplay a media or overlay content segment that has previously not beendisplayed. In another example, the overlay content server 340 cancorrect a sequence of displaying the media segment and the overlaycontent segments when a sequence is out of order. For example, theoverlay content server 340 can determine that the media segment theelectronic device 310 is scheduled to display is third in a sequence ofitems in the overlay content set. The overlay content server candetermine which overlay item in the overlay content set is the first ina sequence for an overlay content set. The overlay content server 340can send the first overlay content segment to the electronic device 310for the electronic device 310 to display. An advantage of correcting asequence of the display of the media segments is that the first mediasegment in the sequence is displayed to the viewer prior to the viewerseeing the second media segment in the sequence. For example, the mediasegments can be sequenced to tell a story for a product beginning withthe first media segment.

In one embodiment, the system 260 can be an input device to receive athreshold number indicating a number of instances a multimedia item canbe displayed. In one example, the input device can be a softwareinterface, such a web portal, on an electronic device, where a user caninput the threshold number of instances at the server 240. In oneexample, the system can receive a multimedia item and threshold numberassociated with it, where the threshold number indicated a maximumnumber of times the multimedia item can be displayed on a displaydevice. In one embodiment, the system 260 can include a memory device tostore the multimedia item and the associated threshold number. Inanother embodiment, the system 260 can be coupled to a memory device tostore the multimedia item and the associated threshold number. Thememory device can be coupled to the system 260 and the server 240.

In one example, the user can set rules or criteria for the server 240 touse to set the threshold number of instances for a multimedia item. Inanother embodiment, the server 240 can include an input device toreceive a threshold number indicating a number of instances a multimediaitem can be displayed.

FIG. 5 illustrates a diagram of a method 500 for determining when toreplace a multimedia item that has previously been displayed on anelectronic device 510 according to one embodiment. The method 500 may atleast partially be performed by processing logic that includes hardware(e.g., circuitry, dedicated logic, programmable logic, microcode, etc.),software (e.g., instructions executed by a processing device), firmwareor a combination thereof. The method 500 may be performed by processinglogic of the electronic device 210 (illustrated in FIG. 2), theprocessing logic of the system 220 (illustrated in FIG. 2), or theprocessing logic of the server 240 (illustrated in FIG. 2).

Referring to FIG. 5, the method 500 begins with determining, byprocessing logic of the electronic device 510, a multimedia item ID fora multimedia item to be displayed on the electronic device 510 (512).The method can also include sending, by the processing logic of theelectronic device 510, a message to the AD server 540 (514). The messagecan include the multimedia item ID and a UID associated with theelectronic device 510 to display the multimedia item. The method caninclude searching, by processing logic of the AD server 540, the recorddatabase 550 for the multimedia item ID associated with the UID (516).The method can include determining, by the processing logic of the ADserver 540, a number of instances that the multimedia item ID associatedwith the UID is stored in the record database 550 (518). The method caninclude determining, by the processing logic of the AD server 540, thatthe number of instances does not exceed a threshold number of instances(520). In one example, the threshold number of instances can be set by asystem administrator or an advertiser when the multimedia item isuploaded or stored in a database coupled to an ad management system(AMS). In another example, different threshold number of instances canbe set for different multimedia items.

The method can include sending, from the AD server 540, a notificationinstructing the electronic device 510 to display the multimedia item(522). The method can include displaying the multimedia item on adisplay of the electronic device 510 when the notification is received(524). In another embodiment, when the electronic device 510 does notreceive a notification before the multimedia item is scheduled to bedisplayed, the electronic device 510 will display the multimedia item.The method can include sending, to the AD server 540, a notificationthat the multimedia item has been displayed on the electronic device 510(526). The method can include sending, to the electronic device 510, atrigger when the notification has been received (528). The method caninclude displaying the trigger on the electronic device 510 (530). Inone embodiment, the trigger can be an invisible overlay that isdisplayed on the electronic device 510 when the ACR system 220(illustrated in FIG. 2) determines that a particular multimedia item wasdisplayed in the electronic device 510 or is currently being displayedon the electronic device 510. In another embodiment, the trigger a webrequest to a specific endpoint on the server 240 (illustrated in FIG.2). The method can include determining, using an ACR system of theelectronic device 510, that the trigger was displayed (532). The methodcan include sending, to the AD server 540, a notification with themultimedia item ID and the UID when the trigger has been display (534).The method can include incrementing, by the AD server 540, the number ofinstances of the multimedia item ID and the associated UID in the recorddatabase 550 (536). For example, each time the multimedia item isdisplayed on the electronic device 510, the AD server 540 can incrementthe number of instances recorded for the multimedia item ID and theassociated UID in the record database 550.

FIG. 6 illustrates a diagram of a method 600 of checking when amultimedia item has previously been displayed on an electronic device610 according to one embodiment. The method 600 may at least partiallybe performed by processing logic that includes hardware (e.g.,circuitry, dedicated logic, programmable logic, microcode, etc.),software (e.g., instructions executed by a processing device), firmwareor a combination thereof. The method 600 may be performed by processinglogic of the electronic device 210 (illustrated in FIG. 2), theprocessing logic of the ACR system 220 (illustrated in FIG. 2), or theprocessing logic of the server 240 (illustrated in FIG. 2).

Referring to FIG. 6, the method 600 begins with determining, byprocessing logic of the electronic device 610, a first multimedia itemID for a first multimedia item to be displayed on the electronic device610 (612). The method can also include sending, by the processing logicof the electronic device 610, a message to the AD server 640 (614). Themessage can include the first multimedia item ID and a UID associatedwith the electronic device 610. The method can include searching, by theprocessing logic of the AD server 640, the record database 650 for thefirst multimedia item ID associated with the UID (616). The method caninclude determining, by the processing logic of the AD server 640, thata number of instances that the first multimedia item ID associated withthe UID is stored in the record database 650 (618). The number ofinstances associated with the first multimedia item ID and the UID canindicate a number of times the electronic device 610 has displayed thefirst multimedia item associated with the multimedia item ID. The methodcan include determining, by the processing logic of the AD server 640,that the number of instances exceeds the threshold number of instances(620). The method can include identifying, by the processing logic ofthe AD server 640, a second multimedia item to display on the electronicdevice 610 when the number of instances exceeds the threshold number ofinstances for the first multimedia item ID (622).

In one embodiment, the identifying the second multimedia item to displaycan include verifying, by the AD server 640, that a number of instancesin the record database 650 for a second multimedia item ID associatedthe second multimedia item to be display on the electronic device 610does not exceed a threshold number of instances (624). For example, therecord database 650 not including a second multimedia item ID associatedwith the UID for the second multimedia item can indicate that theelectronic device 610 has not previously displayed the second multimediaitem.

In another embodiment, when the number of instances in the recorddatabase 650 for the second the second multimedia item being displayedon the electronic device 610 exceeds the threshold number of instances,the AD server 640 can iteratively search the record database 650 toidentify an nth multimedia item that does not exceed a threshold numberof instances. For example, when the threshold number of instance for thesecond multimedia item is exceeded, the AD server 640 can sequentiallysearch the record database 650 for a multimedia item ID for a thirdmultimedia item in the multimedia content set, a multimedia item ID fora fourth multimedia item in the multimedia content set, a multimediaitem ID for a fifth multimedia item in the multimedia content set, andso forth until the AD server 640 identifies a multimedia item ID for annth multimedia item that does not exceed a threshold number of instance.In this example, the AD server 640 can send the nth multimedia iteminstead of the second multimedia item.

In one example, when the threshold number of instances has been exceedfor each of the n multimedia items, the AD server 640 can send amultimedia item from a different multimedia content set. In thisexample, the content overlay network 200 can notify a user oradministrator that an individual or company associated with the initialmultimedia content set should receive a refund for a payment associatedwith the AD server 640 sending the multimedia items to the electronicdevice 610 for display. In another example, when the threshold number ofinstances for each of the multimedia items has been exceeded, the ADserver 640 can reset a count for each of the number of instances for themultimedia item IDs associated with the multimedia items displayed onthe electronic device 610. When the count has been reset, the AD server640 can start over in a sequence of showing the multimedia items.

The method can include sending, from the AD server 640, the secondmultimedia item to the electronic device 610 (624). The method caninclude displaying the second multimedia item on the electronic device610 (626). In one example, the electronic device 610 can display thesecond multimedia item on the electronic device 610 by overlaying orsuperimposing the second multimedia item over the first multimedia item.The method can include sending, to the AD server 640, a notificationthat includes the second multimedia item ID and the UID when the secondmultimedia has been display (628). The method can include incrementing anumber of instances for the second multimedia item ID and the associatedUID when the notification is received (630). Alternatively, the ADserver 640 can increment the number of instances when the AD server 640sends the second multimedia item to the electronic device 610. Thepreceding example illustrates displaying the second multimedia item onthe electronic device 610. However, this exemplary method is notintended to be limiting. The method 600 can be an iterative method wheremultiple multimedia items or all of the multimedia items in multimediacontent set can be sequentially overlayed over previously displayedmultimedia items. Additionally, an order of the multimedia items inmultimedia content set can be sequential, random, or any other definedorder.

FIG. 4 illustrates a diagrammatic representation of a machine in theexample form of a computer system 400 within which a set ofinstructions, for causing the machine to perform any one or more of themethodologies discussed herein, may be executed. In alternativeembodiments, the machine may be connected (e.g., networked) to othermachines in a LAN, an intranet, an extranet, or the Internet. Themachine may operate in the capacity of a server or an electronic devicein a client-server network environment, or as a peer machine in apeer-to-peer (or distributed) network environment. The machine may be apersonal computer (PC), a tablet PC, a set-top box (STB), a PersonalDigital Assistant (PDA), a cellular telephone, a smartphone, a webappliance, a server, a network router, switch or bridge, or any machinecapable of executing a set of instructions (sequential or otherwise)that specify actions to be taken by that machine. Further, while only asingle machine is illustrated, the term “machine” shall also be taken toinclude any collection of machines that individually or jointly executea set (or multiple sets) of instructions to perform any one or more ofthe methodologies discussed herein.

The computer system 400 may correspond to the ACR fingerprinter server105 of FIG. 1. The computer system 400 may correspond to all or part ofthe content distribution network 100 of FIG. 1 or to all or part of thecontent overlay network 200 of FIG. 2. The computer system 400 maycorrespond to at least a portion of a cloud-based computer system suchas discussed herein.

The computer system 400 includes: a processing device 402; a main memory405, such as a read-only memory (ROM); flash memory; dynamic randomaccess memory (DRAM), such as synchronous DRAM (SDRAM) or DRAM (RDRAM);a static memory 406, such as flash memory or static random access memory(SRAM); and a data storage device 418. The processing device 402, themain memory 405, the flash memory, the dynamic random access memory(DRAM), the static memory 406, and the data storage device 418 cancommunicate with each other via a bus 430.

Processing device 402 represents one or more processing devices. Forexample, one or more processing devices can be a microprocessor, centralprocessing unit, or the like. More particularly, the processing device402 may be a complex instruction set computing (CISC) microprocessor, areduced instruction set computer (RISC) microprocessor, a very longinstruction word (VLIW) microprocessor, a processor implementing otherinstruction sets, or a processor implementing a combination ofinstruction sets. The processing device 402 may also be one or morespecial-purpose processing devices. For example, the special-purposeprocessing devices can be an application specific integrated circuit(ASIC), a field programmable gate array (FPGA), a digital signalprocessor (DSP), network processor, or the like. In one embodiment,processing device 402 may include one or more processing cores. Theprocessing device 402 may be configured to execute the instructions 426of a mirroring logic stored in the main memory 405 for performing theoperations discussed herein.

The computer system 400 may further include a network interface device408 communicably coupled to a network 420. The computer system 400 alsomay include: a video display unit 410, such as a liquid crystal display(LCD) or a cathode ray tube (CRT); an alphanumeric input device 412,such as a keyboard; a cursor control device 414, such as a mouse; asignal generation device 416, such as a speaker; or other peripheraldevices. Furthermore, the computer system 400 may include a graphicsprocessing unit 422, a video processing unit 428, and an audioprocessing unit 432. In another embodiment, the computer system 400 mayinclude a chipset, which refers to a group of integrated circuits, orchips, that is designed to work with the processing device 402. Thechipset may control communications between the processing device 402 andexternal devices. For example, the chipset may be a set of chips on amotherboard that links the processing device 402 to high-speed devices,such as main memory 405 and graphic controllers, as well as linking theprocessing device 402 to lower-speed peripheral buses of peripherals,such as a universal serial bus (USB), a peripheral componentinterconnect (PCI), or industry standard architecture (ISA) buses.

The data storage device 418 may include a computer-readable storagemedium 425 that stores instructions 426. The instructions 426 may embodyany one or more of the methodologies of functions described herein. Theinstructions 426 may also reside, completely or at least partially,within the main memory 405 and/or within the processing device 402during execution thereof by the computer system 400. The main memory 405and the processing device 402 may be computer-readable storage media.

The computer-readable storage medium 425 may store instructions 426utilizing logic and/or a software library containing methods that callthe above applications. While the computer-readable storage medium 425is shown in an example embodiment to be a single medium, the term“computer-readable storage medium” or “computer-readable medium” is notintended to be limiting. The “computer-readable storage medium” or“computer-readable medium” can be a single medium or multiple media,such as a centralized or distributed database, and/or associated cachesand servers, that store the one or more sets of instructions. The term“computer-readable storage medium” can include any medium that iscapable of storing, encoding or carrying a set of instruction forexecution by the machine and that cause the machine to perform any oneor more of the methodologies of the present embodiments. The term“computer-readable storage medium” can include, but not be limited to,solid-state memories, optical media, and magnetic media.

In the above description, numerous details are set forth. It will beapparent, however, to one of ordinary skill in the art having thebenefit of this disclosure, that embodiments may be practiced withoutthese specific details. In some instances, well-known structures anddevices are shown in block diagram form, rather than in detail, in orderto avoid obscuring the description.

Although the embodiments may be herein described with reference tospecific integrated circuits, such as in computing platforms ormicroprocessors, other embodiments are applicable to other types ofintegrated circuits and logic devices. Similar techniques and teachingsof embodiments described herein may be applied to other types ofcircuits or semiconductor devices. For example, the disclosedembodiments are not limited to desktop computer systems or laptops andmay also be used in other devices, such as handheld devices, tablets,other thin notebooks, systems on a chip (SOC) devices, and embeddedapplications. Some examples of handheld devices include cellular phones,Internet protocol (IP) devices, smartphones, digital cameras, personaldigital assistants (PDAs), and handheld personal computers (PCs).Embedded applications can include microcontrollers, digital signalprocessors (DSPs), SoCs, network computers (NetPC), set-top boxes,network hubs, wide area network (WAN) switches, or other systems thatmay perform the functions and operations, as discussed in the proceedingparagraphs.

Although the embodiments described herein may refer to a processor orprocessing device, other embodiments are applicable to other types ofintegrated circuits and logic devices. Similar techniques and teachingsof embodiments may be applied to other types of circuits orsemiconductor devices that may benefit from higher pipeline throughputand improved performance. The embodiments herein are applicable to anyprocessor or machine that performs data manipulations. However, theembodiments are not limited to processors or machines that perform 512bit, 256 bit, 128 bit, 64 bit, 32 bit, and/or 16 bit data operations andmay be applied to any processor and machine in which manipulation ormanagement of data is performed.

The embodiments herein provide examples, and the accompanying drawingsshow various examples for the purposes of illustration. However, theseexamples should not be construed to be limiting, as they are merelyintended to provide examples of embodiments rather than to provide anexhaustive list of all possible implementations of embodiments.

Some portions of the detailed description are presented in terms ofalgorithms and symbolic representations of operations on data bitswithin a computer memory. These algorithmic descriptions andrepresentations are used by those skilled in the data processing arts tomost effectively convey the substance of their work to others skilled inthe art. An algorithm is here and generally, conceived to be aself-consistent sequence of operations leading to a desired result. Theoperations are those requiring physical manipulations of physicalquantities. Usually, though not necessarily, these quantities take theform of electrical or magnetic signals capable of being stored,transferred, combined, compared and otherwise manipulated. It has provenconvenient at times, principally for reasons of common usage, to referto these signals as bits, values, elements, symbols, characters, terms,numbers or the like. The blocks described herein may be hardware,software, firmware, or a combination thereof.

The terms in the preceding paragraphs and similar terms are to beassociated with the appropriate physical quantities and are merelyconvenient labels applied to these quantities. Unless specificallystated otherwise as apparent from the preceding paragraphs and figures,discussions utilizing terms such as “detecting,” “initiating,”“determining,” “continuing,” “halting,” “receiving,” “recording,” or thelike, refer to the actions and processes of a computing system, orsimilar electronic computing device, that manipulates and transformsdata represented as physical (e.g., electronic) quantities within thecomputing system registers and memories into other data similarlyrepresented as physical quantities within the computing system memories,computer system registers, other information storage devices, othertransmission devices, or electronic devices.

The words “example” or “exemplary” are used herein to mean serving as anexample, instance, or illustration. Any aspect or design describedherein as “example” or “exemplary” is not necessarily to be construed aspreferred or advantageous over other aspects or designs. Rather, use ofthe words “example” or “exemplary” is intended to present concepts in aconcrete fashion. As used in this application, the term “or” is intendedto mean an inclusive “or” rather than an exclusive “or.” That is, unlessspecified otherwise, or clear from context, “X includes A or B” isintended to mean any of the natural inclusive permutations. That is, ifX includes A; X includes B; or X includes both A and B, then “X includesA or B” is satisfied under any of the foregoing instances. In addition,the articles “a” and “an” as used in this application and the appendedclaims should generally be construed to mean “one or more” unlessspecified otherwise or clear from context to be directed to a singularform. Moreover, use of the term “an embodiment” or “one embodiment” or“an implementation” or “one implementation” throughout is not intendedto mean the same embodiment or implementation unless described as such.Also, the terms “first,” “second,” “third,” “fourth,” etc. as usedherein are meant as labels to distinguish among different elements andmay not necessarily have an ordinal meaning according to their numericaldesignation.

Embodiments described herein may also relate to an apparatus forperforming the operations herein. This apparatus may be speciallyconstructed for the required purposes, or it may comprise a computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in a non-transitorycomputer-readable storage medium, such as, but not limited to, any typeof disk including floppy disks, optical disks, CD-ROMs andmagnetic-optical disks, read-only memories (ROMs), random accessmemories (RAMS), erasable programmable read only memories (EPROMs),electrically erasable programmable read only memories (EEPROMs),magnetic or optical cards, flash memory, or any type of media suitablefor storing electronic instructions. The term “computer-readable storagemedium” can include a single medium or multiple mediums that store theone or more sets of instructions. The single medium or multiple mediumscan be centralized or distributed databases and/or associated caches andservers. The term “computer-readable medium” can include any medium thatis capable of storing, encoding or carrying a set of instructions forexecution by the machine and that causes the machine to perform any oneor more of the methodologies of the embodiments herein. The term“computer-readable storage medium” can include, but not be limited to,solid-state memories, optical media, magnetic media, or any medium thatis capable of storing a set of instructions for execution by the machineand that causes the machine to perform any one or more of themethodologies of the embodiments herein.

The algorithms and displays presented herein are not inherently relatedto any particular computer or other apparatus. Various computing systemsmay be used with programs in accordance with the teachings herein, or itmay prove convenient to construct a more specialized apparatus toperform the operations. The required structure for a variety of thesesystems will appear from the description in the proceeding paragraphs.In addition, the embodiments herein are not described with reference toany particular programming language. It will be appreciated that avariety of programming languages may be used to implement the teachingsof the embodiments herein.

The preceding description sets forth numerous specific details such asexamples of specific systems, components, methods and so forth, in orderto provide a good understanding of several embodiments. It will beapparent to one skilled in the art, however, that at least someembodiments may be practiced without these specific details. In otherinstances, well-known components or methods are not described in detailor are presented in simple block diagram format in order to avoidunnecessarily obscuring the embodiments herein. Thus, the specificdetails set forth above are merely exemplary. Particular implementationsmay vary from these exemplary details and still be contemplated to bewithin the scope of the present embodiments.

The preceding description is illustrative and not restrictive. Manyother embodiments will be apparent to those of skill in the art uponreading and understanding the preceding description. The scope of theembodiments herein should be determined with reference to the appendedclaims, along with the full scope of equivalents to which such claimsare entitled.

What is claimed is:
 1. A system for managing content displayed by anelectronic device, comprising: a processing device; and a non-transitorycomputer-readable storage medium in communication with the processingdevice, the non-transitory computer-readable storage medium storinginstructions that when executed on the processing device cause theprocessing device to perform operations including: receiving informationindicating a first content item to be displayed by the electronic deviceat a scheduled time; determining, from a record database, a firstdisplay count indicating how many times the electronic device haspreviously displayed the first content item; determining that thedisplay count satisfies a first threshold criterion; performing asequential search of the record database, the record database storingrecords relating to a set of content items, the records includingrespective display counts indicating a number of times that each contentitem in the set has been displayed by the electronic device; during thesequential search of the record database, identifying a replacementcontent item from the set of content items, wherein the replacementcontent item has a second display count that satisfies a secondthreshold criterion, the second display count for the replacementcontent item indicates that the replacement content item has beendisplayed for more than one instance by the electronic device, thereplacement content item is a n^(th) content item in the set of contentitems, and the respective display counts for first through (n−1)^(th)content items from the set of content items do not satisfy respectivethreshold criteria; and responsive to the first display count satisfyingthe first threshold criterion and the second display count satisfyingthe second threshold criterion, causing the electronic device to displaythe replacement content item at the scheduled time to replace the firstcontent item.
 2. The system of claim 1, the operations performed by theprocessing device further include updating, in the record database, thesecond display count in response to the electronic device displaying thereplacement content item.
 3. The system of claim 1, wherein theoperations performed by the processing device further includedetermining, via the electronic device, a behavior from a viewer whilethe electronic device displays media content, and the replacementcontent item is further identified from the set of content itemsaccording to the behavior.
 4. The system of claim 1, wherein the firstthreshold criterion is based at least on one of demographic informationrelating to a viewer of the first content item, or on a time of day. 5.The system of claim 1, wherein the second threshold criterion is basedat least on one of demographic information relating to a viewer of thefirst content item, or on a time of day.
 6. The system of claim 1,wherein the respective display counts for the first through (n−1)^(th)content items indicate that each of the first through (n−1)^(th) contentitems has been displayed by the electronic device for more than oneinstance.
 7. A system for managing content displayed by an electronicdevice, comprising: a content server configured to store, on a firstdata storage device, a set of content items; a record database systemconfigured to store, on a second data storage device, respective displaycounts indicating how many times content items have been displayed bythe electronic device; a content management system including aprocessing device and a non-transitory computer-readable storage medium,the non-transitory computer-readable storage medium storing instructionsthat when executed on the processing device cause the processing deviceto perform operations including: receiving information indicating afirst content item to be displayed by the electronic device at ascheduled time; determining, from the record database, a first displaycount indicating how many times the electronic device has previouslydisplayed the first content item; determining that the first displaycount satisfies a first threshold criterion; performing a sequentialsearch of a set of content items stored by the record database; duringthe sequential search of the record database, identifying a replacementcontent item from the set of content items, wherein the replacementcontent item has a second display count that satisfies a secondthreshold criterion, the second display count for the replacementcontent item indicates that the replacement content item has beendisplayed for more than one instance by the electronic device, thereplacement content item is a n^(th) content item in the set of contentitems, and the respective display counts for first through (n−1)^(th)content items from the set of content items do not satisfy respectivethreshold criteria; and responsive to the first display count satisfyingthe first threshold criterion and the second display count satisfyingthe second threshold criterion, causing the electronic device to displaythe replacement content item at the scheduled time to replace the firstcontent item.
 8. The system of claim 7, further comprising an automaticcontent recognition system configured to identify the first content itemto be displayed by the electronic device, and the processing devicereceives, from the automatic content recognition system, the informationindicating the first content item to be displayed by the electronicdevice at the scheduled time.
 9. The system of claim 8, the operationsperformed by the processing device further include updating, in therecord database, the second display count in response to the electronicdevice displaying the replacement content item.
 10. The system of claim7, wherein the operations performed by the processing device furtherinclude determining, via the electronic device, a behavior from a viewerwhile the electronic device displays media content, and the replacementcontent item is further identified from the set of content itemsaccording to the behavior.
 11. The system of claim 7, wherein the firstthreshold criterion is based at least on one of demographic informationrelating to a viewer of the first content item, or on a time of day. 12.The system of claim 7, wherein the second threshold criterion is basedat least on one of demographic information relating to a viewer of thefirst content item, or on a time of day.
 13. The system of claim 7,wherein the respective display counts for the first through (n−1)^(th)content items indicate that each of the first through (n−1)^(th) contentitems has been displayed by the electronic device for more than oneinstance.
 14. A method for managing content displayed by an electronicdevice, comprising: receiving information indicating a first contentitem to be displayed by the electronic device at a scheduled time;determining, from a record database, a first display count indicatinghow many times the electronic device has previously displayed the firstcontent item; determining that the display count satisfies a firstthreshold criterion; performing a sequential search of the recorddatabase, the record database storing records relating to a set ofcontent items, the records including respective display countsindicating a number of times that each content item in the set has beendisplayed by the electronic device; during the sequential search of therecord database, identifying a replacement content item from the set ofcontent items, wherein the replacement content item has a second displaycount that satisfies a second threshold criterion, the second displaycount for the replacement content item indicates that the replacementcontent item has been displayed for more than one instance by theelectronic device, the replacement content item is a n^(th) content itemin the set of content items, and the respective display counts for firstthrough (n−1)^(th) content items from the set of content items do notsatisfy respective threshold criteria; and responsive to the firstdisplay count satisfying the first threshold criterion and the seconddisplay count satisfying the second threshold criterion, causing theelectronic device to display the replacement content item at thescheduled time to replace the first content item.
 15. The method ofclaim 14, further comprising updating, in the record database, thesecond display count for the replacement content item responsive to theelectronic device displaying the replacement content item.
 16. Thesystem of claim 14, further comprising, via the electronic device, abehavior from a viewer while the electronic device displays mediacontent, and the replacement content item is further identified from theset of content items according to the behavior.
 17. The system of claim14, wherein the first threshold criterion is based at least on one ofdemographic information relating to a viewer of the first content item,or on a time of day.
 18. The system of claim 14, wherein the secondthreshold criterion is based at least on one of demographic informationrelating to a viewer of the first content item, or on a time of day. 19.The system of claim 14, wherein the respective display counts for thefirst through (n−1)^(th) content items indicate that each of the firstthrough (n−1)^(th) content items has been displayed by the electronicdevice for more than one instance.